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Abstract 

We present a search for D°-D° mixing in the decays D° — > .fr + 7r~7r + 7r~ using 230.41b" 1 of data 
collected with the BABAR detector at the PEP-II e + e~ collider at SLAC. Assuming CP conservation, 
we measure the time-integrated mixing rate Rm = (0.019 iolois (stat.) ±0.002 (syst.))%, and Rm < 
0.048% at the 95% confidence level. Using a frequentist method, we estimate that the data are 
consistent with no mixing at the 4.3% confidence level. We present results both with and without 
the assumption of CP conservation. By combining the value of Rm from this analysis with that 
obtained from an analysis of the decays D° — » K + 7r~ir°, we find Rm = (0.020 io'.oio)%> wnere the 
uncertainty is statistical only. We determine the upper limit Rm < 0.042% at the 95% confidence 
level, and we find the combined data are consistent with the no-mixing hypothesis at the 2.1% 
confidence level. 
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1 INTRODUCTION 



Transitions between the flavor eigenstates \D°) and \D°) are called D mixing, which is expected to 
have a very small rate in the Standard Model. Due to significant contributions from long-distance 
effects PP|, an accurate estimate is difficult to obtain, but typical theoretical estimates of the time- 
integrated mixing rate are Rm ~ 0(1O~ 6 -1O~ 4 ). The BABAR collaboration has previously reported 
searches for D mixing in the decays to CP-even eigenstates J2], in the decay D° — ► K + ir~ [2], 
and in semileptonic decays [I]. A recent analysis of the decay D° —* K Jr -K~ir set an upper limit 
Rm < 0.054% at the 95% confidence level with a data sample consistent with no mixing at the 
4.5% confidence level (5j- The most stringent constraints on D-mixing parameters to date have 
been obtained by analyzing the decay D° — > K + tt~ [Hj; the rate is determined to be Rm < 0.040% 
at the 95% confidence level. 

We search for the process \D°) — ► \D°) by analyzing the decay of a particle known to be created 
as a \D°) We distinguish doubly Cabibbo-suppressed (DCS) contributions from Cabibbo- 
favored (CF) mixed contributions by the decay-time distribution in the reconstructed wrong-sign 
(WS) decay D° — > K + Tr~ir + Tr~ . The right-sign (RS) decay D° — ► K^tt + tt + -k^ is a normalization 
mode in this analysis. 

The two mass eigenstates 

\D AB )=p\D°)±q\D°) (1) 

generated by mixing dynamics have different masses {w,a.b) and widths (Tab)i with \p/q\ = 1 if 
CP is conserved in mixing. We parameterize the mixing process with the quantities 

m B -m A r B - 

For a multibody WS decay, the time-dependent decay rate, relative to a corresponding RS rate, is 
approximated by [Hj 

r u\ =RD + ay\JR D (Tt) + (Tt) (3) 

r RS(*) 4 
< a < 1, 

where the tilde indicates quantities that have been integrated over the selected phase-space regions. 
Here, Rd is the integrated DCS branching ratio; y' = ycos5 — xsinS and x' = xcos5 + ysin<5, 
where 5 is an unknown integrated strong-phase difference; a is a suppression factor that accounts 
for strong-phase variation over the region; and T is the mean width. The time-integrated mixing 
rate Rm = (x' 2 +y' 2 )/2 = {x 2 +y 2 )/2 is independent of decay mode and should be consistent among 
mixing measurements. Additionally, while the branching ratio of DCS to CF decays depends on 
position in the Dalitz plot, the mixing rate does not. 

We also search for CP violation in a mixing signal by fitting to the D° — ► K + tt^tt + tt^ and 
~ir + ir 7r + samples separately. We consider CP violation in the interference between the 
DCS and mixed contributions, parameterized by an integrated CP-violating-phase (j), as well as 
CP violation in mixing, parameterized by \p/q\. We assume CP invariance in both the DCS and 
CF decay rates. The substitutions 

±i 

(ay cos <fi ± j3x sin <j>) (4) 



ay 



(x 2 + y 2 



±2 (x 2 + y 2 ) (5) 



8 



are applied to Equation |3J using (+) for T(D° —* K tt + tt tt + )/T(D° — > K 7r + 7r + 7r ) and (— ) for 
the charge-conjugate ratio. The parameter (3 is analogous to a and accounts for net (f> variation. 

2 THE BABAR DETECTOR AND DATASET 

We use 230.4 ftT 1 of data collected with the BABAR detector jg] at the PEP-II e+e~ collider 
at SLAC. Charged particles are detected and their momenta measured by a combination of a 
cylindrical drift chamber (DCH) and a silicon vertex tracker (SVT), both operating within a 1.5 T 
solenoidal magnetic field. A ring-imaging Cherenkov detector (DIRC) is used for charged-particle 
identification. Photon energies are measured with a Csl electromagnetic calorimeter (EMC). We 
use information from the DIRC and energy-loss measurements in the SVT and DCH to identify 
charged-kaon and -pion candidates. The data set includes e + e~ collisions at and 40MeV below the 
T(4:S) resonance. All selection criteria were finalized before searching for evidence of mixing in the 
data. 

3 ANALYSIS METHOD 

We reconstruct the decays D* + — > D°Trf and the charge of the soft pion, Kg, is used to determine 
the flavor of the D° candidate. In order to obtain a pure data sample, selection of D° candidates 
includes a requirement of center-of-mass momentum greater than 2.4 GeV/c and the application of 
strict particle-identification (PID) requirements to the daughters of the D°. We accept decays with 
an invariant mass 1.815 < mK-wn-w < 1.915 GeV/c 2 and an invariant mass difference 0.1396 < Am < 
0.1516 GeV/c 2 , where Am = m/f 7r7rvr7rs — mKmrn- We also require that neither it + -k~ combination 
of candidate D° daughters have an invariant mass within 20 MeV/c 2 of the K° s value given in the 
Review of Particle Physics (RPP) jlUj . This cut suppresses background from the singly Cabibbo- 
suppressed decay D° — ► K + K°tt~ followed by K° — > 7r + 7r~. 

The candidate masses and decay times are derived from a vertex fit. First, the D° and D* + 
decay vertices are determined in separate geometric fits, and the x 2 probability of each fit is required 
to be greater than 0.005. The candidate D* + -dec&y tree is then fit for simultaneously optimal D* + 
and D° decay vertices jllj with the D* + decay vertex constrained to the beamspot region. We 
select events for which the \ 2 probability of this fit is greater than 0.01. From this fit, a D° decay 
time, tx-KiTTT, and uncertainty, at, are calculated using the three-dimensional flight path. The full 
covariance matrix, including correlations between the two vertices, is used in the at estimate. For 
signal events, the mean at is near 0.29 ps; we accept decays with at < 0.5 ps. The world-average 
D° lifetime is 0.41 ps 

To separate correctly reconstructed decays from background, and to distinguish mixing contri- 
butions from DCS contributions, unbinned extended maximum likelihood fits to the data sample 
are performed. Probability density functions (PDFs) are fit in two stages to the distributions 
(mKiriTK, Am, iif7r7r7r)- First, the (mx-Kir-K, Am) plane is considered to discriminate between signal 
and background; optimal PDF parameters are established in these dimensions. Second, a fit to 
tx-K-K-K is performed, retaining the PDF-shape parameters of the previous fit to construct a three- 
dimensional likelihood C 

The signal yields from the fit to the (mKmrn, Am) plane are listed in Tabled A simultaneous fit 
is performed to both the large sample of RS decays and the relatively small sample of WS decays; 
thus, signal shape parameters associated with the WS sample are precisely determined by the RS 
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sample, and all associated systematic uncertainties are suppressed. The fit to these distributions 
is shown for the WS sample in Figure ^a,b). 

Table 1: Signal-candidate yields determined by the two-dimensional fit to the (mKirmr, Am) dis- 
tributions for the WS and RS samples. Uncertainties are those calculated from the fit. 

~ D [> Cand. D [> Cand. " 

WS (1.162 ±0.053) x 10 a (1.040 ± 0.051) x 10 3 
RS (3.511 ± 0.006) x 10 5 (3.492 ± 0.006) x 10 5 



The sources of background remaining in the sample may be characterized by three categories in 
the likelihood fits to data. The background that peaks in the tuk^-k-k distribution is due to correctly 
reconstructed D° decays with a misassociated 7r+; this category has the decay-time distribution of 
the RS signal. Second, remaining combinatorial background is present as a nonpeaking component 
of both distributions. This distribution is empirically described by a Gaussian with a power-law 
tail. The third category is due to correctly reconstructed D* + decays with a misreconstructed D°, 
for which the kaon and a pion have been mistaken for each other. This category has the signal 
lifetime distribution. A three-dimensional likelihood is maximized in a fit to tKirmr, after the shape 
parameters are determined in the two mass distributions. 

The RS PDF is fit to the tKnmr distribution to determine the D° lifetime and the detector- 
resolution parameters. The signal shape of tK nnn is an exponential function convolved with a 
double-Gaussian resolution function. The Gaussians have different widths and means; the width 
of each Gaussian is a scale factor multiplied by at, which is determined for each event. The two 
different scale factors are determined by the fit to the data. We find a D° lifetime consistent with 
the nominal value. 

The WS signal PDF in tKnirn is a function based on Equation convolved with the double 
Gaussians described above. The D° lifetime and resolution scale factors and means, determined 
by the fit to the RS tjc^^ distribution, are fixed. We fit the WS PDF to the tKmnr distribution 
allowing yields and background shape parameters to vary. The fit to the tKmrn distribution is 
shown for the WS sample in Figure He, d). 

4 SYSTEMATIC STUDIES 

We quantify systematic uncertainties by performing the analysis with the following changes, in order 
of decreasing significance: the selection of events based on at, the decay-time resolution function, 
the background PDF shape in the rriKmrn distribution, and the measured D° lifetime value. The 
selection requirement of at may skew the decay-time distribution if there is a correlation between 
the two distributions; it is investigated by moving the selection criterion from 0.5 ps to 0.6 ps. The 
double-Gaussian resolution function is investigated by fixing one of the two scale factors to unity, 
determining the other factor from a fit to the RS data, and performing the decay-time fit to the WS 
data. The PDF used to describe the background contribution in the mg^im distribution is changed 
from an exponential to a second-order polynomial. This change allows some fraction of events to 
be weighted toward background, and so affects the number of events contributing to the mixing 
signal. Finally, the fitted lifetime from the decay-time fit to the RS data is not as accurate as the 
value listed in the RPP |10j : this systematic uncertainty is estimated by setting the lifetime to the 
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Figure 1: Distributions of WS data with fitted PDFs (described in Sec. OJ) overlaid. The mK nn7T 
distribution (a) requires 0.14487 < Am < 0. 14587 GeV/c 2 ; the Am distribution (b) requires 
1.859 < mKwmr < 1.869 GeV/c 2 ; and the tKmrn distribution (c) requires Am cuts as the same 
as (a). The tKnnn distribution (d) requires both cuts from (a) and (b). In each of the above his- 
tograms, the white area beneath the dotted line represents signal events, the light gray represents 
swapped K it^, the medium gray represents misassociated ir^ events, and the dark gray represents 
remaining combinatorial background. 
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value given in the RPP. The combined systematic uncertainties for most quantities in Table |2 are 
smaller than statistical uncertainties by a factor of 5; the systematic uncertainties on (ay' cos 4>) 
and (f3x' sm <j>) do not account for correlated uncertainties between the D° and D° samples, and 
thus are conservatively estimated. 

5 RESULTS 

Table 2: Mixing results assuming CP conservation (D° and D° samples are not separated) and 
manifestly permitting CP violation (D° and D° samples are fit separately). The first listed uncer- 
tainty is statistical, and the second is systematic. 





CP conserved 


CP violation allowed 


Rm 


(0.019 t ° \i ± 0.002)% 


(0.017 ± 0.003)% 






ay' cos <f> 


ay' 


-0.006 toM5 ± 0.001 


-0.006 ±oIooe ± °- 006 
fix' sin (j> 

0.002 ±g;ool ± 0.006 




\p/q\ 1.1 ±*:8 ± 0.1 



The results of the decay-time fit, both with and without the assumption of CP conservation 
in a mixing signal, are listed in Table [2J The statistical uncertainty of a particular parameter is 
obtained by finding its extrema for Aln£ = 0.5; in finding the extrema, the likelihood is kept 
maximal by refitting the remaining parameters. Contours of constant Aln£ = 1.15,3.0, enclosing 
two-dimensional coverage probabilities of 68.3% and 95.0%, respectively, are shown in Figure 

We note that Aln£ as a function of the quantity sign(ay') x Rm is approximately parabolic. 
The two-sided interval —0.048% < sign(ay') x Rm < 0.048% contains 95% coverage probability; 
thus, we quote Rm < 0.048% as our upper limit on the integrated mixing rate under the assumption 
of CP conservation. 

A feature of Aln£ in one dimension is that it changes behavior near Rm = because the 
interference term (linear in t in Equation becomes unconstrained. Therefore, we estimate the 
consistency of the data with no mixing using a frequentist method. Generating 1000 simulated data 
sets with no mixing, each with 76,300 events representing signal and background in the quantities 
{mKmrn, Am, tKnim}, we find 4.3% of simulated data sets have a fitted value of Rm greater than 
that in the observed data set. We conclude that the observed data are consistent with no mixing 
at the 4.3% confidence level. 

We combine the value of Rm from this analysis with that obtained from an analysis of the 
decays D° — > K + tt^tt° [Hj by adding the A In £(Rm) curves from the two separate analyses. The 
A In jC(Rm) curves are shown in Fig 13 We extract a central value and an uncertainty from the 
combined curve using the same procedure as for each individual result. With this method, we find 
R M = (0.020 

io.oio)%' wnere the uncertainty is statistical only. We determine the upper limit 
Rm < 0.042% at the 95% confidence level, and we find the combined data are consistent with the 
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Figure 2: Left: Contours of constant Aln£ = 1.15, 3.0 in terms of the doubly Cabibbo-suppressed 
branching ratio and the time-integrated mixing rate. The upward slope of the contour indicates 
negative interference between DCS and mixed contributions. Right: Contours of constant Aln£ = 
1.15,3.0 in terms of the normalized interference term and the integrated mixing rate, for the D° 
and D° samples separately. The hatched regions are physically forbidden. 
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Figure 3: Aln£ as a function of Rm for separate and combined results of K + ir ir + ir and 
K + TT~ir°. (On the axis, the natural logarithm is denoted log.) 
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no-mixing hypothesis at the 2.1% confidence level, as determined from the AItlC(Rm) curve. 

6 CONCLUSION 

We find that the data used in an analysis of D° — > K + tt~tt + it~ are consistent with the no-mixing 
hypothesis at the 4.3% confidence level. Assuming CP conservation, we measure the time-integrated 
mixing rate R M = (0.019 Iftois (stat.) ± 0.002 (syst.))%, and R M < 0.048% at the 95% confidence 
level. Furthermore, we combine these results with those of a similar analysis of the decays D° — > 
K + iT-ir° 5 ]. From this combination, we find R M = (0.020 ±o.oio (stat.))% and R M < 0.042% at 
the 95% confidence level. The combined data sets are consistent with the no-mixing hypothesis 
with 2.1% confidence. 
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